Input-Output Stability of Recurrent Neural Networks with Time-Varying Parameters

We provide input-output stability conditions for additive recurrent neural networks regarding them as dynamical operators between their input and output function spaces. The stability analysis is based on methods from non-linear feedback system theory and includes the case of time-varying weights, for instance introduced by on-line adaptation. The results assure that there are regions in weight space in which a network operates stably regardless to changes of the weights within the respective region. Bounds on the allowed weight deviations are obtained computationally efficient in the framework of interior point optimization methods for linear matrix inequalities and, under certain conditions, are also valid for convergence of corresponding state-space solutions. We apply the methodology to a non-trivial trajectory learning task where we obtain stability regions large enough to cope with parameter drift in the reference model by means of provably stable on-line adaptation. Keywords—recurrent neural networks, input-output behavior, stability, on-line adaptation, non-linear feedback system